Natural Language and Exploration of an Information Space: The ALFresco Interactive System

In traditional natural language systems, the channel of communication between the user and the system was a narrow and constraining device. In many areas natural Ian guage-based information access requires the possibility for the user of exploring the domain and a system's comfortable habitability. The present work presents a rationale for building intelligent interfaces that combine natural language and hy-permedia as new means for human-computer interaction and in particular gives an outline of a prototype of this kind, built for the exploration of Italian frescoes: the ALFresco interactive system. 1 Introduction Natural language interface design has for a long while been based on the teletype approach. The channel of communication between the user and the system was a narrow and constraining device. On the other hand, certainly not all the necessary techniques for understanding "pure" natural language, especially in the pragmat-ics field, seem to be advanced enough to reproduce this traditional way of communicating. In recent times an innovative approach has emerged [Arens et a/., 1989], that acknowledges that a larger bandwidth communication between user and computer can be established, in particular by exploiting graphics. Intelligent multimodal systems have been developed for interfacing a user with a dynamic process such as the simulation of the operations inside a factory [Co-hen et al, 1989], a simulated battlefield [MacLaughlin and Shaked, 1989] or the activities of U.S. Navy ships [Arens ct ai, 1988J. In such applications, the advantage of integrating multiple media in output is obvious, for example, to explain sequences of operations or to display the status of a complex process. Similarly for input , pointing to images on a screen may individuate the *The various modules of ALFresco were realized by the NLP interface group at. IRST. The group currently includes Many of the ideas reported here have emerged working with them. G. Carenini, M. Ponzi and C. Strapparava have given a particular contribution to the present work. objects involved in some desired action [Wahlster, 1988; Hollan et ai, 1988]. Very recently some projects of mul-timodal information presentation have combined graphics and language in output in a new creative way [Feiner and McKeown, 1990; Wahlster et a/., 1991]. A different and nowadays quite popular technology is hypertext. The generalization of the idea of hypertext to multimediality gives rise to the concept of hypermedia:, "a style of building systems for information representation and management around a network of multime-dia nodes connected …